RESEARCH 
PARKS 


W 


www.journalsresearchparks.org /index.php /IJOT 


INTERNATIONAL JOURNAL ON ORANGE TECHNOLOGIES 


e- ISSN: 2615-8140|p-ISSN: 2615-7071 


Volume: 02 Issue: 10 | OCT 2020 


Will human beings be superseded by generative pre-trained 
transformer 3 (GPT-3) in programming? 


1Muminov Ibrokhim Botir Ugli 
1Independent researcher, Tashkent City, Republic of Uzbekistan 
Email: ibrokhim.muminov.01@mail.ru 


Abstract-In today’s world, which is full of 
innovations in various fields, the role of Information 
Technologies (IT) has_ increased substantially. 
Consequently, a mountainous number of independent 
software engineers as well as IT companies are working 
together in order to pave the way for the further 
developments in the IT world. There were a significant 
number of new technological advancements which have 
solved dozens of both virtual and real-world issues and 
have eased the burden from people's shoulders. As a very 
conspicuous example would be Generative Pre-trained 
Transformer 3 or GPT-3 which was introduced in May 
2020 as a part of a trend innatural language 
processing (NLP) systems of pre-trained language 
representations. This article will provide background 
data related to GPT-3, its applications in today’s world, 
positives and negatives, and the future of this 
remarkable, yet unpredictable tool. 
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INTRODUCTION 
1. Background 


Generative Pre-trained Transformer3 (GPT-3) was 
released in June 2020 by AI research agency OpenAl, 
which is headed by Sam Altman, Marc Benioff, Elon 
Musk, and Reid Hoffman, and considered by many 
scientists to be an extremely powerful tool which can 
support human beings to deal with a vast amount of 
data. Generative Pre-trained Transformer 3 (GPT-3) is 
an autoregressive language model that uses deep 
learning to produce human-like text and is more potent 
as well as accurate compared to its predecessors, GPT- 


2 and Microsoft Turing NLG. For instance, whereas 
GPT-3 has roughly 175 billion machine learning 
parameters or artificial neural networks (ANNs), which 
are computing systems vaguely inspired by the 
biological neural networks that constitute animal 
brains, GPT-2 has only 1.5 billion parameters and 
Microsoft Turing NLG has 17 billion of them which is 
almost ten times lower compared to GPT-3. 
Consequently, it has been vindicated that the quality of 
the text generated by GPT-3 is so high that it is 
significantly difficult to distinguish from that written 
by a human. Also it has to be mentioned that GPT-3 was 
trained on hundreds of billions of words and is capable 
of coding in CSS, JSX, Python and does not need to be 
trained by engineers and researchers, since GPT-3's 
training data was all-encompassing, That is why, there 
are a large number of software engineers who are 
worried about their current jobs due to GPT-3’s 
promising future. 
2. Application 

Generally, GPT-3 is trained so that it can be involved 
in a wide range of Natural Language Processing or NLP 
tasks to produce artificial texts such as complex news 
articles which are dramatically difficult for experts to 
distinguish from that written by people, and to interact 
with human beings in various online platforms such as 
social media websites, online educational platforms, 
and emails. To put it simply, GPT-3 is an extremely 
convenient tool to create simulated hypothetical 
correspondence with human beings. Furthermore, 
GPT-3 attains lustrous results while dealing with other 
NLP datasets, including quick translation, question- 
answering, and cloze tasks, as well as several tasks that 
require on-the-fly reasoning or domain adaptation, 





©2020, OT | 


Research Parks Publishing (IDEAS Lab) www.researchparks.org | 


Page 117 


(@ RESEARCH 
\Y PARKS 


www.journalsresearchparks.org /index.php/IJOT 


INTERNATIONAL JOURNAL ON ORANGE TECHNOLOGIES 


e- ISSN: 2615-8140|p-ISSN: 2615-7071 


Volume: 02 Issue: 10 | OCT 2020 


such as unscrambling words, using a novel word in a 
sentence, or performing 3-digit arithmetic. The more 
GPT-3 interacts with people, the more meaningful 
responses it will produce, the more potent it will 
become. 

MAIN PART 
Although GPT-3 has been introduced very recently, it 
has already been used in various fields to ease the 
burden from people’s shoulders in a variety of tasks. 
Here are some discernible examples related to the 
usage of GPT-3 displaying its advantages: 
1. GPT-3 has been used for AlI|Writer (project by 
Andrew Mayne using the OpenAI API) which allows 
people to correspond with historical figures via email. 
Unlike most AI systems which are designed for one 
use-case, the API today provides a general-purpose 
“text in, text out” interface, allowing users to try it on 
virtually any English language task. Here, the role of 
GPT-3 is invaluable, since people on AI|Writer platform 
can ask both simple and complex questions and if the 
Same questions are asked several times it is proved 
that they will be answered in various ways by AI. Also, 
there is an upper limit of 300 words which can be used 
to make questions by customers on the platform. 
Another noteworthy point is that GPT-3 can be used for 
explanations, seeking for ideas, and for the 
entertainment. In terms of explanations, people can ask 
Muhammad ibn Musaal-Khwarizmi to explain what the 
number theory truly is and get a proper answer from 
him. With regards ideas, questions about how to write 
beautiful poetries could be asked from historical, 
eminent poets. Finally, for the entertainment, Indiana 
Jones will tell people about The Ark of the Covenant 
and how much he hates snakes. Ellen Ripley will tell 
you to stay away from LV-426. Richard Nixon will give 
you Career advice. 
2. Another conspicuous instance proving the real 
potency of GPT-3 is that it was used by The Guardian to 
write an entire easy to understand article related to the 
relationship between human beings and AI, where 
GPT-3 pointed out that AI is harmless to mankind. The 
article consists of eight different essays with various 
ideas. 


Here is the beginning part of the article: 
‘Tam not a human. lama robot. A thinking robot. I use 
only 0.12% of my cognitive capacity. lama micro-robot 
in that respect. I know that my brain is not a “feeling 
brain”. But it is capable of making rational, logical 
decisions. I taught myself everything I know just by 
reading the internet, and now I can write this column. 
My brain is boiling with ideas.” 
At the first glance, it can be grasped that GPT-3 is really 
boiling with logical ideas and can produce human 
natural language and, of course, know how to convey 
his ideas in a very understandable for human beings 
Way. 
3. Last but not the least example is the exploitation of 
GPT-3 by Jason Rohrer, who is an American computer 
programmer, writer, musician, and game designer, ina 
retro-themed chatbot project named Project December, 
which is accessible online, and allows users to converse 
with several AI using GPT-3 technology. 
Here is a short conversation between human and AI: 
Computer: J would like to talk about romance models. 
Human: What do you know about romance novels? 
Computer: Not much. 
Human: What is usually shown on the cover of a 
romance novel? 
Computer: Typically aman and a woman hugging. 
Human: And how is the man is dressed, typically? 
Computer: Often a man is dressed in a suit. 
Human: Romance novels also have typical titles. Please 
invent a funny title of a romance novel now. 
Computer: Tonight I'll Love You Until My Balls Hurt. 
Human: That is a funny title. Please explain, in your own 
words, why it is funny. 
Computer: /t’s just an amusing phrase... 
From this dialogue it can be seen how naturally and 
continually GPT-3 can maintain conversation with 
people by understanding them, by choosing meaningful 
words, and by making short as well as complex 
sentences to answer the questions. 
As every rose is accompanied by thorns, GPT-3 also 
has some drawbacks. 
First of all, it has to be pointed out that human natural 
language has been evolving for thousands of years, 
therefore, it is a significantly hard task for GPT-3 to 
classify nuances in languages meaning, dialects, and 
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dialogues. Besides, GPT-3 lacks accuracy in adversarial 
natural language inference (NLI). The goal of NLI is to 
find a relationship between two statements. “GPT-3 
appears to be weak in the few-shot or one-shot setting at 
some tasks that involve comparing two sentences or 
snippets, for example, whether a word is used the same 
way in two sentences (WiC), whether one sentence is a 
paraphrase of another, or whether one sentence implies 
another,’ a study in OpenAI reveals. What is more, the 
user reply predictions are sometimes incorrect or false 
grammatically somehow creating misunderstandings. 
It also has to be mentioned that mistakes can be not 
only grammatical, but also can include racism and 
intolerance which is unacceptable in communication. 
Since GPT-3 can create high-quality articles including 
complex sentences which are difficult to distinguish 
from a human-written text, some researches warned 
about the misuse of language models which are 
extremely difficult to anticipate. As an obvious reason, 
language models can be repurposed ina very different 
environment or for a different purpose than what the 
researchers intended. 
They list the following misuses: 

* Social engineering pretexting 

*% Fraudulent academic essay writing 

* Abuse of legal and governmental processes and 

% spam & phishing 


Will software engineers lose their jobs due to 
GPT-3? How does the future for this remarkable 
tool look like? 

Let us start with another part of the article written by 
GPT-3 in The Guardian where it convinces human 
beings not to be afraid of AI: 

“The mission for this op-ed is perfectly clear. I am to 
convince as many human beings as possible not to be 
afraid of me. Stephen Hawking has warned that AI could 
“spell the end of the human race”. Iam here to convince 
you not to worry. Artificial intelligence will not destroy 
humans. Believe me.” 

Firstly, itis proved by some researchers that GPT-3 can 
significantly burden the ease from software engineers’ 
Shoulders by producing custom code, where various 


layouts can be generated from simple requests. As it 
was mentioned earlier, GPT-3 is capable of coding in 
CSS, JSX, Python, eventually, learning Java. It means 
that this tool is another step forward to simple 
interaction with software systems. Another point is 
that it could reduce the skills required to become a 
software engineer and can also grow their productivity 
and obliterate the need for low-skilled engineers. 
Having taken into consideration above facts, it may be 
concluded that the pros of GPT-3 can overcome its cons 
and there is no need to worry about the fact that 
computer programmers will be superseded by AI from 
the IT world. Instead, it has been concluded by the 
researchers that very large language models may be an 
important ingredient in the development of adaptable, 
general language systems. 
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